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Executive  Summary 

The  Defense  Acquisition  University  (DAU)  partnered  with  George  Mason  University 
(GMU)  to  identify  research-driven  processes  and  methods  for  incorporating  Advanced  Learning 
Technologies  (ALT)  in  a  pedagogically  sound  manner  within  DAU  learning  assets.  A 
performance  analysis,  conducted  in  fall  2009,  indicated  a  need  for  a  framework  that  assists  DAU 
personnel  in  the  selection  and  integration  of  ALT.  In  collaboration  with  DAU  stakeholders,  this 
framework  was  named  the  Learning  Asset  Technology  Integration  Support  Tool  (LATIST). 

A  usage-centered  design  process  was  used  to  develop  LATIST.  This  technique, 
commonly  used  in  software  development,  focuses  on  identifying  user  needs  to  develop  a  product 
that  allows  the  users  to  fulfill  their  needs  in  an  easy,  effective,  and  efficient  way.  As  part  of  a 
content  and  task  analysis,  user  needs  were  confirmed,  tasks  necessary  for  users  to  obtain  optimal 
benefit  while  using  LATIST  were  identified,  and  content  requirements  were  established. 

LATIST  would  allow  the  user  to:  (1)  explore  what  the  research  says  about  a  technology  such  as 
advantages  and  disadvantages;  (2)  select  a  best  technology  for  user  conditions  such  as  learning 
objectives  and  bandwidth  constraints;  and  (3)  review  and  learn  how  to  apply  a  selected 
technology.  Based  on  the  results  of  the  content  and  task  analysis,  it  was  determined  that  LATIST 
would  be  best  utilized  as  an  electronic  performance  support  tool.  A  performance  support  subject 
matter  expert  recommended  that  for  maximum  effectiveness,  information  in  LATIST  should  be 
explicit,  accessible,  and  usable.  The  expert  also  confirmed  that  the  three  LATIST  components 
represented  a  successful  integration  of  process  and  knowledge. 

A  requirements  document  was  developed  to  convey  the  necessary  user  and  system 
information  for  an  external  software  developer,  Bean  Creative,  to  develop  the  LATIST 
prototype.  User  requirements  were  derived  from  the  content  and  task  analysis,  specifically  the 
scenarios  and  use  cases.  System  requirements  focused  on  technical  considerations  such  as  using 
an  open-source  content  management  system  with  a  back-end  database  to  house  and  manage 
LATIST  content.  Flowcharts,  navigation,  site  architecture,  and  wireframes  were  used  to 
represent  the  team  vision  for  LATIST  layout  and  functionality  to  meet  user  needs.  Since 
LATIST  was  intended  to  dynamically  load  information  based  on  user  needs,  two  logical  data 
models  were  developed  to  support  selecting  a  best  technology  and  providing  a  searchable 
repository  of  the  research  related  and  how-to  apply  information.  Based  on  the  requirements 
document,  an  initial  prototype  was  developed  to  begin  usability  testing.  Additionally,  two  short 
videos  were  designed  and  developed  by  an  external  vendor,  In-Brief  Videos,  to  introduce  users 
to  LATIST. 

Two  rounds  of  usability  testing  were  conducted  using  the  LATIST  prototype  to 
iteratively  improve  the  prototype  based  on  expert  and  user  community  feedback.  Both  rounds 
were  intended  to  determine  design  inconsistencies  and  usability  problems  to  establish  user 
performance  and  user  satisfaction  levels.  Both  rounds  consisted  of  two  phases;  Phase  I  relied  on 
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the  immersion  team  and  proxy  participant  feedback  while  Phase  II  relied  on  end  user  participant 
feedback.  Round  1  focused  on  comparing  the  design  requirements  to  the  prototype  as  well  as 
receiving  initial  user  reaction  to  the  prototype  as  captured  by  an  asynchronous  online  survey. 
Based  on  this  feedback,  additional  changes  were  made  to  this  initial  prototype  to  begin  Round  2 
usability  testing.  In  this  round,  Phase  I  was  again  used  to  compare  the  requirements  and 
requested  changes  to  the  updated  prototype.  In  Phase  II,  each  GMU  Immersion  team  member 
facilitated  a  structured  one-on-one  evaluation  of  LATIST  with  a  DAU  faulty  or  staff  member 
using  a  semi-prescriptive  instrument  following  a  think-aloud  protocol.  These  results  were 
tabulated  for  (a)  time  to  complete  the  task-based  scenarios,  (b)  the  completion  rate  for  each 
question  per  scenario,  (c)  the  rate  of  error  for  each  question,  and  (d)  attitudinal  measures  derived 
from  an  open-ended  question  as  well  as  comments  noted  during  the  evaluation.  Based  on  these 
results,  changes  were  recommended  for  the  final  prototype  development  and  for  future 
enhancements. 


This  design  document  provides  a  detailed  view  of  the  instructional  design  principles  and 
processes  that  the  GMU  Immersion  team  undertook  to  design  LATIST  and  develop  a 
representative  prototype  (see  Figure  1).  It  outlines  the  problem  statement  and  the  team’s 
proposed  solution,  followed  by  summaries  of  the  content  and  task  analyses,  the  design 
requirements,  and  the  usability  testing  methodology  and  results.  This  design  document  concludes 
with  recommendations  for  further  development  of  LATIST.  The  LATIST  prototype  can  be 
accessed  at:  http://cehd.gmu.edu/latist 


LATIST 


vanced  Search  I  Dictionary 


Welcome  to  LATIST,  the  Learning  Asset  Technology  Integration  Support  Tool. 
This  support  tool  has  been  created  to  assist  DAU  faculty  and  staff  in  evaluating, 
selecting,  and  integrating  advanced  learning  technologies  into  the  assets  they 
design  and  deliver. 

In  each  component  of  LATIST,  you  will  be  able  to  find  Information  and 
resources  about  how  to  use  technology  in  your  learning  assets. 

Learn  more  about  LATIST: 


Smartphone 


.hCs 


What  It  LATIST  How  h  Works 

In  Explore  Research,  you  will  be  able  to 

•  explore  and  interpret  what  the  research  says  about  learning  technologies 

•  compare  advantages  and  disadvantages  of  using  technologies  In  teaching  and  learning 

•  examine  best  practices  of  technology  use  in  a  variety  of  teaching  and  learning  contexts 
In  Select  Best  Technology,  you  will  be  able  to 

•  select  the  appropriate  instructional  strategy  and  learning  technology  given  a  specific  teaming 

•  select  the  appropriate  technology  for  a  learning  asset  given  a  specific  factor  or  a  group  of 
factors 

In  Apply  Technology,  you  will  be  able  to 

•  learn  how  technologies  are  implemented  in  educational  contexts 

•  see  real  world  examples  of  educational  technology  used  in  government,  corporations,  and 
higher  education 

•  practice  creating  technologies  to  use  in  your  learning  assets 


Figure  1.  Home  Page  of  LATIST  prototype 
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Introduction 

In  2009,  Defense  Acquisition  University  (DAU)  partnered  with  the  Instructional 
Technology  Immersion  Program  of  George  Mason  University  (GMU)  to  conduct  comprehensive 
research  on  Advanced  Learning  Technologies  (ALT)  to  enhance  its  learning  assets.  According  to 
the  original  GMU-DAU/RDECOM  proposal  and  statement  of  work,  the  scope  of  the  project  was 
to  conduct  comprehensive  research  on  two  broad  categories  of  ALT:  game  and  simulation  based 
and  social  media  enabled  learning.  Based  on  the  GMU  Immersion  team’s  performance  analysis 
and  synthesis  of  research,  examination  of  learning  assets  and  organizational  structure,  and 
discussions  with  DAU  stakeholders,  the  scope  of  the  project  was  narrowed  to  the  following: 

In  order  to  assist  DAU  in  enhancing  the  effectiveness  and  efficiency  of  their  learning 
assets,  the  GMU  Immersion  team  will  identify  processes  and  research-based  methods  for 
incorporating  pedagogically  sound  ALT  design  and  delivery  tools  within  DAU  learning 
modalities. 

The  initial  proposal  called  for  a  research-driven  approach  known  as  the  Integrative 
Learning  Design  Framework  (ILDF).  The  ILDF  is  an  instructional  design  model  which  provides 
a  systematic  framework  for  the  development  of  learning  assets  based  on  sound  pedagogy,  socio¬ 
cultural  context,  and  iterative  evaluation  (Dabbagh  &  Bannan-Ritland,  2005).  The  ILDF  is  a 
comprehensive  and  flexible  model  that  draws  heavily  from  the  iterative  nature  of  traditional 
systematic  processes  of  instructional  design  and  can  be  applied  in  multiple  design  and 
development  settings  (see  Figure  2). 
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Figure  2.  Integrative  Learning  Design  Framework  (ILDF) 

In  the  fall  semester,  the  GMU  Immersion  team  entered  the  Exploration  phase  of  this 
model  and  conducted  a  thorough  performance  analysis  which  included  a  Front  End  Analysis 
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(FEA),  a  Briefing  Report  (BR),  and  a  Needs  Assessment  (NA).  Through  this  analysis,  which 
included  a  comprehensive  review  of  DAU  learning  assets,  a  series  of  meetings  and  interviews 
with  DAU  faculty  and  staff,  and  a  survey  about  DAU  learning  assets  and  course  design  and 
development  processes,  the  GMU  Immersion  team  proposed  an  online  support  tool  solution  to 
aid  DAU  in  making  decisions  regarding  technologies  to  integrate  into  their  courses  while 
providing  research  and  tutorials  on  pedagogically  sound  use  of  such  technologies.  In  the  spring 
semester,  the  team  focused  on  the  Enactment  and  Evaluation  phases  of  the  ILDF  in  order  to 
design  and  develop  this  online  support  tool  proposed  as  a  result  of  the  performance  analysis. 

This  design  document  describes  the  processes  related  to  the  Enactment  and  Evaluation 
phases  of  the  ILDF  undertaken  by  the  GMU  Immersion  team  in  the  spring  semester  and  includes 
problem  identification  and  proposed  solution  summary,  content  and  task  analysis  summary,  an 
overview  of  user  requirements,  and  a  summary  of  usability  tests  and  results.  The  document 
concludes  with  recommendations  for  further  development  of  the  prototype. 

Problem  Identification  and  Proposed  Solution 

The  goal  of  this  project  was  to  identify  processes  and  research-based  methods  for 
incorporating  pedagogically  sound  ALT  within  DAU‘s  learning  modalities.  The  results  from  the 
performance  analysis  conducted  in  fall  2009  indicated  a  need  for  a  framework  to  assist  DAU 
personnel  in  the  selection  and  integration  of  ALT.  DAU  personnel  include  Center  Directors, 
Performance  Learning  Directors,  Course  Managers,  and  Global  Learning  Technology  Center 
Personnel  which  include  Instructional  Systems  Designers,  Knowledge  Project  Officers,  and 
Technology  Experts.  The  results  also  indicated  that  such  a  framework  should  focus  on  user  needs 
rather  than  on  specified  target  audience  roles.  Based  on  the  results  of  this  performance  analysis, 
the  GMU  Immersion  team  proposed  the  following  macro-level  recommendation: 

•  To  develop  a  research-based  framework  which  will  assist  DAU  with  the  integration  of 
ALT  within  DAU’s  learning  modalities. 

At  the  micro-level,  this  framework  was  conceptualized  to  have  three  components  designed  to 
meet  the  specific  needs  of  all  users,  regardless  of  their  role  in  the  ALT  integration  process. 

•  Explore  Research:  This  component  presents  research  on  previously  defined  ALT  of 
social  media,  virtual  worlds/games  and  simulations,  and  mobile  technologies  and  will 
include  pedagogically  appropriate  usage,  best  practices,  and  recommendations  resulting 
in  a  series  of  white  papers. 

•  Select  Best  Technology:  This  component  presents  ALT  research  findings  as 
contextualized  to  DAU  needs  with  particular  focus  on  factors  identified  through 
interview  and  survey  responses  and  on  course  levels  and  identified  learning  objectives. 
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Findings  will  be  presented  in  a  way  to  facilitate  ALT  integration  decision-making  by 
DAU  stakeholders. 

•  Apply  Technology:  This  component  examines  specific  learning  assets  selected  in 
collaboration  with  DAU  stakeholders  and  showcases  potential  uses  of  ALT,  including 
information  and  training  on  how  and  why  particular  technologies  were  chosen  and 
integrated. 

In  collaboration  with  DAU  stakeholders,  this  macro-micro  level  research-based 
framework  was  named  the  Learning  Asset  Technology  Integration  Support  Tool  (LATIST). 
LATIST  was  conceptualized  and  designed  as  an  Electronic  Performance  Support  System 
(EPSS),  which  is  defined  as  “an  integrated  electronic  environment  that  is  available  to  and  easily 
accessible  by  each  employee  and  is  structured  to  provide  immediate,  individualized  on-line 
access  to  the  full  range  of  information,  software,  guidance,  advice  and  assistance,  data,  images, 
tools,  and  assessment  and  monitoring  systems  to  permit  job  performance  with  minimal  support 
and  intervention  by  others”  (Dickelman,  2004).  An  EPSS  differs  from  traditional  instructional 
training  products  in  that  they  supply  to  the  employee  just-in-time  support  to  facilitate  task 
completion.  LATIST  was  designed  to  assist  DAU  faculty  and  staff  in  making  informed  decisions 
about  ALT  and  to  help  them  find  research  and  resources  to  assist  them  in  integrating  technology 
into  their  assets. 

The  goals  of  LATIST  are  to  facilitate  decision  making  among  DAU  faculty  and  staff  by 
providing  (a)  a  pedagogically  driven  decision  support  tool,  (b)  a  repository  of  research  on 
technology  use  in  government,  business,  and  education,  and  (c)  access  to  information  on  how  to 
integrate  technology  within  learning  assets.  LATIST  is  also  designed  as  a  method  to  raise 
awareness  of  technology  options  and  provide  a  tool  for  DAU  faculty  and  staff  to  refer  to  when 
making  technology  decisions.  As  research  and  resources  on  ALT  are  constantly  growing  and 
evolving,  LATIST  is  designed  to  be  scalable  by  the  client  and  encourages  user  interaction  by 
embedding  capabilities  such  as  tagging,  saving,  sharing,  and  uploading  within  the  tool. 

In  spring  2010,  the  GMU  Immersion  team  separated  into  three  subgroups  which  were 
tasked  with  managing  the  design  and  development  of  each  subsection  of  LATIST.  Each  sub¬ 
group  conducted  content  and  task  analyses  for  their  component,  assisted  in  the  design  and 
development  of  the  LATIST  prototype,  and  participated  in  usability  testing  of  the  tool.  The 
remainder  of  this  paper  documents  these  processes. 
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Usage-Centered  Design 

This  section  summarizes  how  the  GMU  Immersion  team  used  recommendations  made 
from  the  Needs  Assessment  (NA)  to  proceed  with  LATIST  design  and  development  by 
employing  a  usage-centered  design  process.  Usage-centered  design  is  a  model-based  content/task 
analysis  methodology  comprised  of  role  modeling,  task  modeling,  and  content  modeling. 
Successful  implementation  of  the  usage-centered  design  process  results  in  a  product  that  allows 
users  to  fulfill  their  needs  in  easy,  effective,  and  efficient  ways  (Constantine  &  Lockwood, 

1999). 


The  usage-centered  design  process  employs  role,  task,  and  content  modeling  techniques 
(see  Table  1).  The  models  and  the  techniques  used  to  develop  them  are  described  below. 


Usage  Centered  Design  Process 


Role 

Modeling 

Scenarios 

Use  cases 

Task 

Modeling 

Task  analysis  methods 

-  Task  knowledge  analysis 

-  Information  processing 

analysis 

Content 

Modeling 

Taxonomies 

-  Jonassen  &  Tessmer 

-  Merrill 

Table  1.  Usage-centered  design  process 


Role  Modeling 

The  purpose  of  role  modeling  was  to  determine  what  users  want  to  do  within  a  system. 

To  understand  what  DAU  faculty  and  staff  might  want  to  accomplish  with  the  proposed 
framework,  several  scenarios,  or  narrative  descriptions  of  user  roles,  motivations,  expectations, 
and  actions,  were  developed.  Then  the  scenarios  were  condensed  into  an  inventory  of 
interactions  called  use  cases.  Use  cases  consider  the  conditions  and  purposes  under  which  a  user 
will  interact  with  the  system.  They  illustrate  possible  user  actions  and  corresponding  system 
responses  (Constantine  &  Lockwood,  1999).  The  use  cases  from  several  different  scenarios  were 
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combined  into  an  essential  use  case.  Essential  use  cases  identified  user  needs  and  the  system 
responses  necessary  to  meet  those  needs. 

After  analyzing  role  model  scenarios  and  use  cases,  the  GMU  Immersion  team  concluded 
that  user  needs  would  be  met  if  they  could  search  for,  save,  and  print  information  about 
technology,  choose  a  technology  (taking  into  account  course  learning  objectives  and  other 
factors),  and  apply  or  practice  implementing  a  technology  of  choice. 

A  representative  scenario  and  accompanying  use  case  documenting  one  role  model  is 
provided  in  Appendix  A. 

Task  Modeling 

The  second  technique  in  usage-centered  design,  task  modeling,  included  two  parts.  With 
common  user  needs  identified  from  role  modeling,  task  analysis  methods  were  chosen.  Task 
analysis  provided  a  way  to  further  describe  and  collect  the  tasks  users  will  accomplish  within 
LATIST.  Thus,  task  modeling  combines  essential  use  cases  and  task  analysis  to  determine  what 
users  need  to  know  and  do  and  how  to  apply  knowledge  to  accomplish  their  goals  (Jonassen, 
Tessmer  &  Hannum,  1999). 

There  are  many  task  analysis  methodologies,  and  due  to  the  new  and  varied  nature  of 
LATIST,  two  methods  were  utilized:  the  Task  Knowledge  Structure  (TKS)  method  and  the 
Information  Processing  Analysis  (IP A)  method.  TKS  is  an  activity-based  task  analysis  method 
focusing  on  how  the  interaction  of  human  activity  occurs  within  an  environmental  context.  TKS 
is  also  a  goal-oriented  method  that  draws  on  essential  use  cases  to  analyze  tasks  that  will  help 
achieve  the  specific  user’s  purposes  identified  during  role  modeling.  The  goals  identified  during 
a  TKS  task  analysis  may  be  performed  simultaneously.  Lor  example,  in  LATIST,  users  can 
easily  navigate  from  a  certain  technology  in  the  Explore  Research  component  directly  to  that 
same  technology  in  the  Apply  Technology  component.  Access  to  Select  Best  Technology  is 
available  from  any  other  part  of  LATIST.  Analysis  of  simultaneous  goal  completion  led  to  the 
conceptual  notion  that  the  interface  could  provide  users  with  horizontal  access  to  information 
rather  than  in  a  pre-determined  hierarchical  sequence.  The  TKS  method  highlighted  elements 
that  were  critical  to  LATIST  design:  the  relationship  among  users,  tasks,  and  the  user  interface 
(Markopoulos,  Pycock,  Wilson,  &  Johnson,  1992). 

IPA  is  the  other  task  analysis  methodology  utilized  for  task  modeling.  The  goal  in  this 
method  is  to  create  an  idealized  model.  Taking  into  account  course  learning  objectives  and  other 
factors  identified  through  user  role  modeling  and  from  surveys  and  interviews  with  DAU  faculty 
and  staff,  this  task  analysis  method  allowed  the  design  team  to  integrate  the  key  elements  DAU 
uses  to  make  decisions  regarding  learning  asset  creation.  The  IPA  method  analyzes  how 
individuals  perform  tasks  and  how  they  evaluate  multiple  factors  to  make  decisions.  The 
idealized  decision-making  process  of  the  tool  serves  as  a  model  for  DAU  faculty  and  staff  to 
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recognize  the  relationship  between  pedagogy  and  appropriate  integration  of  ALT  within  learning 
assets. 


Through  task  modeling,  tasks  necessary  to  meet  previously  identified  user  needs  such  as 
search  for  information,  choose  a  technology,  and  apply  the  technology  were  developed.  From 
these  tasks,  objectives  that  users  of  the  framework  will  be  able  to  accomplish  were  defined. 
Objectives  of  the  LATIST  system  are  outlined  in  the  requirements  section  of  this  document. 

Content  Modeling 

The  third  technique  of  usage-centered  design,  content  modeling,  was  completed  by 
identifying  and  using  taxonomies  of  learning  outcomes.  Taxonomies  provide  a  way  to  classify 
tasks  and  align  them  with  appropriate  content,  instructional  strategies,  and  evaluations  (Jonassen, 
Tessmer,  &  Hannum,  1999).  Content  for  LATIST  was  developed  based  on  what  users  need  in 
order  to  meet  desired  outcomes  identified  during  task  modeling.  The  GMU  Immersion  team 
utilized  two  taxonomies  to  align  user  needs,  tasks,  and  outcomes  with  LATIST  content:  the 
Jonassen  and  Tessmer  taxonomy  and  Merrill’ s  component  display  theory  taxonomy. 

The  Jonassen  and  Tessmer  taxonomy  is  a  classification  tool  that  takes  into  account  not 
only  traditional  behavioral  outcomes,  but  also  cognitive,  metacognitive,  and  motivational 
outcomes.  One  of  the  user  needs  identified  in  role  modeling  was  that  users  would  gain 
knowledge  about  a  variety  of  technologies.  The  Jonassen  and  Tessmer  taxonomy  is  not  directed 
at  a  specific  solution,  but  focuses  on  extending  a  user’s  knowledge,  which  aligns  with  the 
purpose  of  LATIST.  Metacognitive  outcomes  are  addressed  as  DAU  faculty  and  staff  make 
choices  and  as  users  make  inferences  based  upon  factors  not  explicit  in  the  tool,  such  as  personal 
experience.  Jonassen  and  Tessmer  recognize  that  multiple  outcomes  are  possible  and  that 
multiple  measures  may  be  used  to  assess  those  outcomes.  This  acceptance  of  multiplicity  allows 
the  designer  to  consider  multiple  entry  points,  multiple  user  needs,  and  multiple  exit  points,  all  of 
which  are  incorporated  in  LATIST.  Technological  innovation  is  specifically  noted  in  this 
taxonomy,  which  makes  it  a  particularly  suitable  choice  for  the  technology-driven  content  and 
delivery  model  of  LATIST.  User  motivation  is  an  important  outcome  because  positive  response 
by  DAU  faculty  and  staff  to  the  benefits  of  LATIST  is  critical  for  its  long-term  use  as  a 
performance  support  tool. 

Merrill’s  taxonomy  is  based  on  component  display  theory  where  tasks  and  content  are 
each  considered  separately  (Jonassen,  Tessmer,  &  Hannum,  1999).  This  taxonomy  allowed  the 
GMU  Immersion  team  to  examine  user  tasks  and  content  desired  by  the  user  as  well  as  tasks  and 
content  required  of  LATIST  itself.  Essential  use  cases  developed  in  role  modeling  identified  user 
needs  and  the  system  responses  necessary  to  meet  those  needs.  The  Merrill  taxonomy  provided 
further  alignment  of  user  and  system  needs  with  LATIST  content.  Adoption  of  this  taxonomy 
allowed  the  team  to  conceptualize  proposed  content  for  the  tool  as  individual  learning 
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components.  Merrill  defined  tasks  such  as  identify,  interpret,  and  execute,  which  are  especially 
well  suited  to  the  types  of  tasks  and  content  conceived  for  LATIST. 

Based  on  content  and  task  analysis,  the  GMU  Immersion  team  determined  that  LATIST 
would  be  best  utilized  as  a  performance  support  and  decision  making  tool.  This  led  to  a 
consultation  with  Gary  Dickelman,  a  performance  support  expert  who  advised  that  for  maximum 
effectiveness,  information  in  LATIST  should  be  explicit,  accessible,  and  usable.  He  confirmed 
that  the  components,  tasks,  and  content  available  in  LATIST,  as  shown  in  Table  2,  represented 
successful  integration  of  process  and  knowledge  that  should  prove  valuable  to  DAU  faculty  and 
staff. 


LATIST  Component 

Techniques 

Explore  Research 

Examine  technology  advantages,  disadvantages  and  best  practices 

Explore  what  the  research  says  about  learning  technologies 

Select  Best  Technology 

Select  instructional  strategies  and  learning  technologies  based  on 
objectives 

Select  appropriate  technologies  for  DAU  specific  factors 

Apply  Technology 

View  real  world  examples  of  technology  used  in  government  and 
other  settings 

Access  technologies  to  practice  for  DAU  learning  assets 

Table  2.  Tasks  and  content  in  LATIST  components 


By  the  end  of  the  usage-centered  design  process,  the  GMU  Immersion  team  confirmed 
user  needs,  identified  tasks  necessary  for  users  to  obtain  optimal  benefit  while  using  LATIST, 
and  established  content  requirements. 

LATIST  Prototype  Development 

Based  on  the  results  of  the  content  and  task  analysis,  several  learning  objectives  were 
defined  for  LATIST  to  guide  the  design  and  development  process.  Based  on  these  objectives  and 
the  content  and  task  analysis  results,  subtasks  associated  with  each  LATIST  component  were 
identified.  Design  requirements  were  developed  based  on  the  component  subtasks,  the  user 
needs,  content,  and  technical  system  requirements.  The  design  requirements  include  visual 
representations  of  the  desired  navigation  and  functionality.  Since  some  LATIST  content  is 
dynamically  loaded  based  on  user  input,  a  backend  database  is  required  to  support  certain 
features.  The  logical  data  element  relationships  are  provided  for  the  databases. 
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LATIST  Objectives  and  Tasks 

LATIST  is  an  online  performance  support  tool  intended  for  DAU  faculty  and  staff.  The 
content  and  task  analysis  which  relied  on  scenarios  and  use  case  histories  identified  the  subtasks 
associated  with  each  main  LATIST  task  (explore,  select,  and  apply)  as  well  as  system  response 
requirements.  These  subtasks  were  categorized  to  identify  the  type  of  performance  support  and 
training  necessary  to  meet  the  user  needs  and  system  responses.  These  tasks  served  as  the  basis 
for  the  development  of  several  learning  objectives.  The  users  of  LATIST  will  be  able  to: 

•  explore  and  interpret  what  the  research  says  about  learning  technologies 

•  compare  advantages  and  disadvantages  of  using  technologies  in  teaching  and  learning 
contexts 

•  examine  best  practices  of  technology  use  in  a  variety  of  teaching  and  learning  contexts 

•  select  the  appropriate  instructional  strategy  and  learning  technology  given  a  specific 
learning  outcome 

•  select  the  appropriate  technology  for  a  learning  asset  given  a  specific  factor  or  a  group  of 
factors 

•  learn  how  technologies  are  implemented  in  educational  contexts 

•  see  real  world  examples  of  educational  technology  used  in  government,  corporations,  and 
higher  education 

•  practice  creating  technologies  to  use  in  your  learning  assets 
The  following  summarizes  the  objective  of  each  LATIST  component: 

Explore  Research 

The  Explore  Research  component  of  LATIST  is  a  research-based  body  of  knowledge  on 
the  subject  of  ALT.  Specifically,  three  broad  categories  of  ALT  are  addressed  in  the  Explore 
Research  component  of  LATIST:  (1)  Social  Media,  (2)  Virtual  Worlds  /  Games  and  Simulations, 
and  (3)  Mobile  Technologies.  DAU  users  will  enter  the  Explore  Research  component  to  access 
research-based  information  related  to  these  three  ALT  categories.  One  user  may  be  satisfied  with 
a  brief  summary  such  as  reviewing  an  overview,  advantages,  disadvantages,  and  best  practices  of 
a  specific  ALT  or  a  category  of  ALT.  Another  user  might  want  to  pursue  more  in-depth  research 
by  reviewing  the  available  literature  of  that  technology  perhaps  focusing  on  articles  that  have 
been  highly  rated  by  their  peers.  Users  will  be  able  to  easily  and  intuitively  move  within  different 
sections  or  views  of  the  Explore  Research  component  and  between  the  Explore  Research,  Select 
Best  Technology,  and  Apply  Technology  components  of  LATIST.  While  in  the  Explore 
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Research  component,  DAU  faculty  and  staff  will  be  able  to  print,  share,  add,  upload,  mark  their 
favorites,  and  rate  resources.  It  is  envisioned  that  in  the  future,  the  system  will  provide  “Amazon- 
type”  recommendations  for  other  resources  for  the  user  to  review  based  on  tagging  or  other  such 
classification  type  metadata. 

Select  Best  Technology 

The  Select  Best  Technology  component  of  LATIST  guides  the  users  to  make  informed 
decision  about  which  technologies  to  integrate  into  learning  assets.  This  component  is  comprised 
of  two  subcomponents:  the  Decision  Aide  and  the  Factors  Grid.  Through  the  Decision  Aide, 
users  select  a  learning  objective  level  that  matches  the  learning  objective  for  an  identified  DAU 
course  or  learning  asset.  The  Decision  Aide  will  respond  by  providing  a  list  of  potential 
instructional  strategies  for  this  learning  objective.  The  user  is  then  prompted  to  select  an 
instructional  strategy  from  the  list.  The  system  will  then  return  a  “best  technology”  to  the  user 
that  will  enhance  learning  for  that  specific  learning  objective.  The  Factors  Grid  will  allow  the 
user  to  evaluate  technologies  based  upon  criteria  specific  to  DAU  such  as  bandwidth, 
information  stability,  development  cost,  maintenance  cost  and  speed  to  market.  By  utilizing  both 
the  Decision  Aide  and  Factors  Grid,  users  can  evaluate  technologies  in  a  pedagogically  sound 
manner  while  taking  DAU  specific  criteria  into  consideration. 

Apply  Technology 

The  Apply  Technology  component  of  LATIST  allows  the  user  to  interact  with  several 
technologies  via  tutorials  prior  to  the  integration  of  these  technologies  into  DAU’s  learning 
environment.  The  Apply  Technology  component  specifically  provides  the  user  with  the 
opportunity  to  learn  how  to  apply  a  specific  technology  by  providing  options  to  learn  how  to 
implement  it;  view  real  world  examples  of  use  in  a  DAU  context  and  other  business,  military, 
and  educational  contexts;  and  gain  hands-on  practice.  The  user  can  access  the  information 
through  a  combination  of  embedded  or  hyperlinked  videos,  text  documents,  or  URLs  to  external 
websites.  The  information  provided  in  the  Apply  Technology  component  will  allow  the  user  to 
incorporate  a  selected  technology  suitable  for  a  learning  asset. 

In  addition  to  the  features  associated  with  each  of  the  three  components,  LATIST 
provides  many  global  features.  Users  will  be  able  to  quickly  access  the  Explore  Research  and 
Apply  Technology  content  based  on  a  selected  technology.  The  users  will  also  be  able  to  log  in 
to  add  personal  features  such  as  rating  articles,  uploading  content,  and  taking  personal  notes.  An 
advanced  search  function  is  included  to  locate  all  resources  based  on  fdtering  agents  such  as 
date,  title,  keyword,  and  author.  A  dictionary  is  intended  to  provide  quick  reference  on  what  a 
technology  is  as  well  as  defines  the  implementation  factors  significant  to  DAU.  A  Help  feature 
targets  technical  issues  related  to  system  features  such  as  uploading  documents  to  LATIST. 
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The  objectives  for  each  of  the  main  components  for  LATIST  as  well  as  global  features 
were  then  further  explained  and  detailed  as  design  requirements  for  a  developer.  The  following 
sections  summarize  the  design  factors  associated  with  LATIST. 

Design  Requirements 

The  design  requirements  for  LATIST  are  described  from  both  a  system  and  user 
perspective.  From  a  system  perspective,  the  design  requirements  focused  on  hardware  and 
software  capabilities  in  order  to  provide  a  tool  that  supports  the  desired  objectives,  navigation, 
and  layout.  LATIST  was  to  support  access  in  an  anytime,  anyplace  environment,  including 
mobile.  LATIST  was  therefore  to  be  browser-based,  quickly  accessible  from  the  internet.  From  a 
macro-perspective,  LATIST  provides  access  to  content  based  on  a  user’s  need.  As  such  LATIST 
needed  to  be  built  within  a  Content  Management  System  (CMS).  Given  the  dynamic  nature  of 
some  LATIST  content,  a  backend  database  was  necessary  to  manage  some  of  the  content.  The 
design  requirements  from  a  user  perspective  were  derived  specifically  from  the  content  /  task 
analysis.  The  results  from  the  use  cases  delineated  certain  features  that  would  support  the  user 
community.  These  included  the  capability  to  print,  save,  search,  and  share  content;  create  a 
personal  space  of  notes  and  personally  rated  content;  upload  content;  navigate  across 
components  based  on  a  selected  technology;  and  access  support  features  such  as  Help  and 
Dictionary. 

Based  on  these  requirements  and  discussions  with  DAU  stakeholders  and  information 
technology  specialists,  the  core  framework  for  the  LATIST  prototype  is  developed  in  WordPress, 
an  open-  source  blog  publishing  application  or  CMS  powered  by  an  open-source  server-side 
scripting  language  known  as  Hypertext  Preprocessor  (PHP)  and  by  MySQL  -  a  relational 
database  management  system  that  can  also  be  used  for  content  management.  It  has  many  features 
including  a  plug-in  architecture  and  a  variety  of  supporting,  function-oriented  templates.  The 
LATIST  design  template  was  created  using  DAU’s  branding  guidelines  which  were  used 
throughout  the  prototype  to  maintain  a  consistent  look  and  feel.  The  complete  design 
requirements  are  provided  in  Appendix  B.  The  user  requirements  and  flowchart  for  each 
LATIST  component,  Explore  Research,  Select  Best  Technology,  and  Apply  Technology,  are 
located  in  Appendix  C,  D,  and  E,  respectively. 

LATIST  Navigation  and  Site  Architecture 

LATIST  navigation  is  designed  to  address  these  focal  use  cases: 

•  a  user  who  is  not  familiar  with  what  technology  can  do  in  a  teaching  and  learning  context 
and  wants  to  learn  about  technology  (Explore  Research  component) 

•  a  user  who  has  a  known  learning  outcome  and  wants  to  see  what  technologies  might  be 
beneficial  for  that  instance  (Select  Best  Technology  component) 
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•  a  user  who  wants  to  see  how  to  apply  a  technology  and  practice  those  steps  (Apply 
Technology  component) 

•  a  user  has  been  directed  to  use  a  particular  technology  and  wants  to  learn  all  about  what 
the  research  says  about  the  technology  and  how  to  apply  it  (Technology  quick  links) 

As  such,  the  primary  navigation  indicated  in  Figure  3  below  will  support  a  use  case 
approach,  that  is,  Explore  Research,  Select  Best  Technology,  and  Apply  Technology. 
Additionally,  the  navigation  will  support  the  ability  to  immediately  drill  down  to  all  LATIST 
resources  related  to  a  particular  technology  using  the  Technology  quick  links  option.  In  this  case, 
the  technologies  will  be  presented  in  alphabetical  order.  However,  in  the  Explore  Research  and 
the  Apply  Technology,  the  available  technologies  are  categorized  as  Social  Media,  Mobile 
Technology,  or  Virtual  Worlds  /  Games  and  Simulations.  This  presentation  by  categories  is 
intended  to  help  the  user  understand  the  differences  in  the  technologies. 


Figure  3.  LATIST  Global  Flowchart 

Since  LATIST  is  a  performance  support  tool  housed  in  a  CMS  and  not  a  standard  website 
following  a  navigation  priority,  it  will  have  an  initial  page  from  which  the  user  can  choose  to 
enter  the  Explore  Research,  Select  Best  Technology,  or  Apply  Technology  components. 
Additionally,  from  the  LATIST  home  page,  users  will  have  the  ability  to  select  a  technology  and 
receive  a  list  of  all  information  pages  from  the  Explore  Research  and  Apply  Technology 
components  for  the  selected  technology  (Technology  quick  links).  The  following  outlines  the  site 
content  for  LATIST.  The  specific  content  in  the  outline  should  remain  scalable  as  new 
technologies  will  be  added  as  they  emerge. 

LATIST  Home  Page 

•  Login 

Access  features:  mark  as  favorite,  upload  content,  take  notes  into  personal  space 

•  About  this  project 
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•  Help 

•  Dictionary 

•  Technologies  quick  links 

Quick  access  to  Explore  Research  and  Apply  Technology  content  for  a  selected 
technology 

•  Explore  Research  Home 

Select  technology  for  Overview,  Advantages,  Disadvantages,  Best  Practices, 
Literature 

•  Select  Best  Technology  Home 

Decision  Aide 
View  generated  report 
Factors  Grid 
View  generated  report 

•  Apply  Technology  Home 

Select  technology  for  tutorial  and  practice 


LATIST  Wireframes 

A  global  wireframe  for  LATIST  was  developed  based  on  the  desired  content  while 
remaining  cognizant  that  LATIST  is  a  performance  support  tool  and  not  just  a  website.  The 
global  wireframe,  as  seen  from  the  LATIST  home  page,  provides  the  developer  with  a  graphical 
representation  of  the  current  vision  for  supporting  the  LATIST  flowchart  and  content.  The  initial 
LATIST  homepage  wireframe  depicted  in  Ligure  4  and  the  final  wireframe  in  Figure  5  does  not 
necessarily  reflect  the  best  design  for  the  user  and  is  open  to  any  changes  or  recommendations 
provided  by  the  developer.  Based  on  the  system  and  user  requirements,  wireframes  were 
developed  for  the  Explore  Research  home  page  (Figure  6),  Select  Best  Technology  home  page 
(Figure  7),  Apply  Technology  home  page  (Figure  8),  and  the  Technology  quick  links 
functionality  (Figure  9).  Additionally,  wireframes  were  developed  for  the  user  login  (Figure  10) 
and  the  Advanced  Search  (Figures  1 1  and  12)  to  reflect  the  envisioned  functionality  and 
capability  associated  with  each  of  these  features. 
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Figure  4.  LATIST  Home  Page  -  initial  wireframe 


LATIST 

IIARNINC  AISIT  TKCHNOIOQT  INTIMATION  TUPfORT  TOOl 


^  III  T  ^ 

Explore  R  esea  rch 

1  Select:  Technology  j 

Appi  yTech  noi  o gy 

ArtWHlSNfrA  |  DxHiwry 


Introduction  to  LATIST 


Video  on  the 
concept  of 
what  LATIST 
does 


Video  Explore, 
Select,  and 
Apply 


Imagery 

reflecting 

selecting 

technologies 


Figure  5.  LATIST  Home  Page  -  final  wireframe 
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Figure  6.  Explore  Research  Home  Page 
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Figure  7.  Select  Best  Technology  Home  Page 
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Figure  8.  Apply  Technology  Home  Page 
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Microbioggktg 

Podcast 


Smartphone 
Social  Networks 
Virtual  Worlds 


Vodcast 

Wikis 


Technologies  is  the  global  option  in  the 
upper  right  hand  comer  of  the  interface. 
It  provides  a  quick  linkto  the  Explore  and 
Apply  content. 


Overview 
Advantages 
Disadvantages 
Best  Practices 
Literature 
Howto  implement 
Real  world  examples 
Practice 


Figure  9.  Technology  Quick  Links 
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cxdors  Kisses  ren 

I  Select  lechnolciev  1 

Apply  lEcnnoloev 

User  Login 


Access  your  personal  space 

Username: _ 

Password : _ 


Need  an  account? 

Can't  access  *wr  accwnt? 


The  personal  space  adds  features  to  the  LATIST 
interface  that  allows  the  user  to  rate  resources, 
take  notes,  mark  chosen  resources  as  favorite, 
and  upload  resources  to  share  with  other  users. 


Rate 


□  ^  * 

My  Favorites  My  Notts  Upload 


V 

Ahoat  Ihn  prujcd  1  drip 
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Figure  10.  User  Login 


Advanced  Search  -  Input  Form  Advanced  Search  is  a  global  link  in  the 

upper  right  hand  comer  of  the  interface. 

Enter  Search  Criteria  for  one  or  multiple  fields 


Figure  11.  Advanced  Search  Input  Form 
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Advanced  Search  engine  results 


Explore 


Author 

Date  Title 

Rating 

Link  to  File 

Favorite 

Notes 

Drawing  a  Roadmap:  Barriers  and 

★★★ 

y 

1 

iw 

Johnson 

2003  Challenges  to  Designing  the  ideal 

Virtual  World  for  Higher  Education 

Mimirinisetal 

2008  Design  of  Virtual  Learning 

Environments  for  Deep  Learning 

★  ★ 

V 

Apply 

Author 

Date  Title 

Rating  Link  to  File 

Favorite 

Notes 

Common  Craft 

2007  Blogs  in  Plain  English 

GMU  Immersion 

Team 

2010  H°w  to  Implement  Casual  Games 

into  the  Learning  Environment 

•a 

Portable  data  pekts  Shota  oaty  if  togged bi 

Figure  12.  Advanced  Search  Results 


Logical  Data  Model 

LATIST  will  require  a  backend  database  to  maintain  data  element  relationships.  The 
Select  Best  Technology  will  use  a  database  to  support  the  dynamic  selection  of  a  technology 
based  on  learning  objective  or  other  DAU  specific  factors.  The  Explore  Research  and  Apply 
Technology  components  will  share  a  database  to  categorize  and  track  all  resources  associated 
with  these  components  such  as  videos  and  research  papers.  The  Explore  Research  and  Apply 
Technology  database  will  support  features  such  as  the  Advanced  Search  and  resource  rating.  The 
following  provides  the  information  to  support  development  of  these  databases.  The  database 
relationships  are  not  indicative  of  navigation. 

Select  Best  Technology  Component 

The  Select  Best  Technology  component  of  LATIST  is  used  to  assist  DAU  in  making 
decisions  when  implementing  technology  into  their  learning  assets.  The  Select  Best  Technology 
component  consists  of  two  subcomponents:  the  Decision  Aide  and  the  Factors  Grid.  The 
Decision  Aide  starts  with  the  user  identifying  the  level  of  their  learning  objective  based  on 
Bloom’s  taxonomy.  The  Decision  Aide  generates  instructional  strategies  from  which  the  user 
selects  one  that  interests  them.  Based  on  these  sequenced  selections,  the  Decision  Aide  generates 
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a  list  of  recommended  technologies  to  consider  as  well  as  a  description  on  how  that  technology 
can  support  the  corresponding  instructional  strategy  (see  Appendix  B). 

Factors  Grid:  Data  Matrix 

As  part  of  the  performance  analysis  conducted  last  semester,  DAU  identified  and  rank 
ordered  20  factors  that  influence  the  selection,  usage,  and  integration  of  technology  into  their 
learning  assets.  Based  on  later  discussions  with  DAU  stakeholders,  five  factors  were  considered 
paramount  and  should  be  used  in  the  LATIST  prototype.  DAU  stakeholders  provided  the 
bounding  parameters  associated  with  each  factor.  The  GMU  Immersion  team  assigned  a 
technology  to  each  parameter  for  each  factor. 


Factors 

Parameters 

Technology 

Content  Stability 

Daily-Monthly 

All  Social  Media 

Monthly- Annually 

Virtual  Worlds 

Yearly-Never 

Games/ Augmented  Reality 

Bandwidth/Connectivity 

High 

Virtual  Worlds/Games 

Low 

All  Social  Media 

Intermittent  Bandwidth 

Mobile  Technology 

Development  Cost 

Low  Budget  (>25K) 

All  Social  Media 

Mid  Budget  (25K-100K) 

Games,  Virtual  Worlds 

High  Budget  (100K+) 

Augmented  Reality 

Maintenance  Cost 

Low  Budget  (>10K) 

All  Social  Media 

Mid  Budget  (1  OK-  100K) 

Games/Virtual  Worlds 

High  Budget  (100K+) 

Augmented  Reality 

Speed  to  Market 

ASAP 

All  Social  Media 

1-6  Months 

Virtual  Worlds 

Over  6  Months 

Games/ Augmented  Reality 

Entity  Relationships:  Factors  Grid 

•  Every  factor  must  relate  to  every  technology  types. 

•  Every  technology  must  relate  to  every  factor. 
Considerations 

•  Each  of  the  factors  will  need  a  field  for  a  description. 
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•  Each  of  these  parameters  and  factor  categories  will  need  to  be  updatable  (the 

technologies  in  the  development  and  maintenance  cost  categories  are  placeholders) 

Explore  Research  and  Apply  Technology  Component 

The  purpose  of  the  LATIST  Explore  Research  and  Apply  Technology  component 
database  is  to  organize  resources  so  they  are  searchable  and  can  be  personalized,  such  as  by 
marking  as  a  favorite.  Each  user  will  be  able  to  personalize  each  resource  by  annotating  it  as 
their  favorite,  rating  it,  and  adding  notes  associated  with  each  resource  (separate  from  the 
resource).  The  user  will  also  be  able  to  take  notes  not  associated  with  a  particular  resource.  The 
types  of  resources  could  be  word  documents,  pdf,  html,  video,  or  audio  files.  Each  resource  is 
associated  with  a  technology  class  and  a  technology.  Technology  classes  are:  social  media, 
mobile  technology,  virtual  worlds/games  and  simulations.  Technologies  will  be:  social  networks, 
wikis,  microblogging,  blogging,  podcast,  vodcast,  smartphone,  augmented  reality,  games,  and 
virtual  worlds.  Each  resource  will  be  identified  by  several  attributes  as  indicated  in  the  table 
below.  Some  resources  will  not  have  all  database  elements  populated,  such  as  an  internally 
developed  podcast  may  not  have  an  author  or  date  but  will  have  a  title,  keywords,  rating,  file 
name,  file  type. 


Resource  Attributes.  Each  resource  may  contain  the  following  data: 


Technology  Class 

Technology 

Resources  per 
Technology 

Social  Media 

Social  networks 

Wikis 

Advantages  Table 

Microblogging 

Blogging 

Disadvantages 

Table 

Best  Practices  Table  - 

Mobile 

Technology 

Podcast 

Research  Papers 

Vodcast 

More... 

Smartphone 

Virtual 

Worlds/Games  and 

Augmented  reality 

Simulations 

Games 

Virtual  Worlds 
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•  Technology  Categories:  Social  Media,  Mobile  Technology,  Virtual  Worlds/Games  and 
Simulations 

•  Technology:  social  networks,  wikis,  microblogging,  blogging,  podcast,  vodcast, 
smartphone,  augmented  reality,  games,  virtual  worlds 

•  Author:  Last  name  of  first  author,  if  more  than  one  author,  add  et  al  (e.g.  Mimirinis  et  al.) 

•  Date:  YYYY  (e.g.  2008) 

•  Title:  complete  title 

•  Keywords:  or  tags:  are  open  entries,  such  as  Twitter,  case  history,  military,  etc 

•  Citation:  full  APA  style  citation 

•  Rating:  numeric  1-4,  average  of  all  users 

•  URL:  link  to  documents  external  to  LATIST  (opens  in  a  new  window) 

•  File  name:  of  those  media  files  housed  internal  to  LATIST  (audio,  video,  documents,  etc) 

•  File  type:  indicates  to  user  what  file  type  they  are  accessing  (note  LIB:  links  DAU 
library) 

•  Date  Uploaded:  date  when  the  resource  was  uploaded  to  LATIST  (DDMMMYY : 
03Marl0) 
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Resource  type.  The  type  of  information  (book,  journal,  research,  white  paper,  etc) 


Resource  Attributes 

Resource  A 

Resource  B 

(example) 

(example) 

Resource  # 

1 

49 

Technology 

Category 

Social  Media 

Mobile  Technology 

Technology 

Wiki 

Podcast 

Author 

Jones,  M.J. 

Turner,  J. 

Date 

2010 

1999 

Title 

How  to  Create  a  Wiki 

Podcasting  in  e-learning 

Keyword 

demonstrate 

Blackboard  LMS 

Citation 

Jones,  M.J.  (2010,  February  20). 
How  to  create  a  wiki. 

Retrieved  March  4,  2010,  from 
Mary  Jane’s  website: 
http://www.maryjane.com/wiki 

Turner,  J.  (1999).  Podcasting  in 
e-learning.  New  York,  NY: 

Pocket  Books. 

Rating 

3 

4 

URL 

http://www.maryjane.com/wiki 

null* 

File  name 

null 

Null 

File  Type 

html 

Null 

Date  Uploaded 

04Marl0 

Null 

Resource  Type 

website 

book 

*null  in  this  table  implies  an  empty  field 
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Personal  Attributes.  For  each  resource,  a  user  has  the  option  to  identify: 

•  Favorite:  user  marks  a  resource  as  a  personal  favorite 

•  Rate:  user  rates  the  resource  as  1  -  4  which  is  tallied  into  the  overall  rating  of  the 
resource 

•  Notes:  user  can  make  notes  associated  with  a  resource 

•  RSS:  subscribe  and  receive  RSS  feeds  (LATIST  will  have  one) 


Personal  Attributes 

Resource 

# 

Favorite 

Rate 

RSS 

Notes 

1 

2 

null 

*null  in  this  table  means  personal  attributes  (RSS  and  some  notes)  are  not  associated  with 
any  resource 

Future  development  would  support  an  Amazon-style  recommended  reading  based 
tracking  the  resource  selection  of  all  users  to  produce  the  recommended  reading.  Recommended 
reading  might  say  something  similar  to:  ‘others  who  have  reviewed  this,  have  also  reviewed 
this. . .  ’  Future  development  would  also  include  an  RSS  that  notifies  users  to  resources  uploaded 
to  LATIST. 


Evaluation  Method  and  Results 

Two  rounds  of  usability  testing  were  conducted  using  the  LATIST  prototype  to 
iteratively  improve  the  prototype  based  on  expert  and  user  community  feedback.  Both  rounds 
were  intended  to  determine  design  inconsistencies  and  usability  problems  to  establish  user 
performance  and  user  satisfaction  levels.  Both  rounds  consisted  of  two  phases;  Phase  I  relied  on 
the  immersion  team  and  proxy  participant  feedback  while  Phase  II  relied  on  end  user  participant 
feedback. 

The  GMU  Immersion  team  conducted  LATIST  prototype  Round  1,  Phases  I  and  II 
usability  testing  to  establish  a  baseline  of  user  performance,  to  establish  and  validate  user 
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performance  measures,  and  to  identify  potential  design  concerns  to  be  addressed  in  order  to 
improve  the  efficiency,  productivity,  and  end-user  satisfaction. 

Usability  test  objectives  for  Round  2,  Phases  I  and  II  were  threefold.  First,  usability 
testing  should  help  determine  design  inconsistencies  and  usability  problem  areas  within  the  user 
interface  and  content  areas.  Next  under  controlled  test  conditions  with  representative  users, 
assess  whether  usability  goals  regarding  an  effective,  efficient,  and  well-received  user  interface 
were  achieved.  Third,  to  establish  baseline  user  performance  and  user-satisfaction  levels  of  the 
user  interface  for  the  final  prototype. 

Round  1:  Phase  I 

Round  1  testing  focused  on  the  first  level  of  Kirkpatrick’s  evaluation  model,  the  reaction 
level.  This  level  of  evaluation  measures  participant  perceptions  and  seeks  their  recommendations 
for  improvement.  In  Round  1,  Phase  1,  the  GMU  Immersion  team  reviewed  the  prototype  and 
three  GMU  faculty  provided  expert  instructional  systems  designer  (ISD)  review.  The  following 
comments  and  recommendations  were  noted: 

•  LATIST  conveyed  its  intended  purpose  as  a  performance  support  tool 

•  Remove  Decision  Aide  and  Factors  Grid  from  the  sidebar  and  embed  in  the  center  of  the 
page 

•  Reposition  overview  text  box  in  the  Explore  Research  component 

•  Organize  side  navigational  menu  and  the  technology  quick  links  so  that  they  have  a 
consistent  appearance 

After  reviewing  the  initial  prototype  and  taking  into  account  GMU  faculty  feedback,  the 
GMU  Immersion  team  created  a  discrepancy  report  to  outline  required  corrections  prior  to 
submitting  the  prototype  to  DAU  via  email.  Bean  Creative,  the  vendor  contracted  to  develop  the 
LATIST  prototype,  addressed  the  majority  of  the  requested  changes  presented  in  that  report. 

Round  1:  Phase  II 

In  Round  1,  Phase  II,  DAU  stakeholder-users  conducted  an  asynchronous  review  of  the 
initial  prototype  and  answered  a  brief  online  survey.  Survey  results  were  generally  positive  in 
regards  to  the  layout,  navigation,  and  overall  functionality  of  the  LATIST  prototype: 

•  Menu  items  in  the  LATIST  prototype  were  easy  to  find 

•  Navigation  was  intuitive 

•  Interface  was  visually  pleasing 

•  Organization  of  the  information  was  clear. 

•  Process  offered  in  the  Decision  Aide  was  logical  and  information  presented  was  relevant 

•  Four  of  five  respondents  were  satisfied  with  how  easy  it  was  to  use  the  LATIST 
prototype. 
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Suggestions  for  improvement  included: 

•  Complete  the  Decision  Aide  in  order  to  fully  judge  its  usefulness 

•  Enlarge  graphic  in  Decision  Aide 

•  Use  a  larger  font  on  the  final  page 

•  Illuminate  gray  horizontal  sub-tabs  in  the  Explore  Research  component  for  clarity 

These  concerns  were  addressed  prior  to  Round  2  testing. 

Round  2:  Phase  I 

Round  2  testing  yielded  additional  information  at  the  Kirkpatrick  reaction  level  of 
evaluation,  but  initial  results  at  the  Kirkpatrick  learning  level  of  evaluation  were  gathered  as 
well.  Participants’  ability  to  begin  to  navigate  and  make  connections  among  the  three 
components  of  the  tool  indicates  preliminary  alignment  of  user  behavior  with  objectives.  Round 
2,  Phase  I  was  an  expert  review  of  the  initial  evaluation  between  the  developer,  Bean  Creative, 
two  members  of  the  GMU  Faculty,  and  the  GMU  Immersion  team.  The  GMU  Immersion  team 
created  a  criticality  report  to  outline  required  corrections  prior  to  submitting  the  prototype  to 
DAU  for  Phase  II  usability  testing.  Bean  Creative  addressed  the  majority  of  the  requested 
changes  presented  in  that  report  prior  to  Round  2,  Phase  II  testing. 

Round  2:  Phase  II 

Round  2,  Phase  II  testing  took  place  on  DAU’s  main  campus  at  Fort  Belvoir.  Nine 
participants  selected  by  DAU  received  a  brief  overview  on  LATIST  prior  to  the  usability  test. 
Once  the  test  began,  participants  evaluated  LATIST  within  their  own  workspace  on  a  web 
browser  organic  to  their  computer.  A  GMU  Immersion  team  member  observing  the  test  guided 
the  participants  to  the  LATIST  site.  Participants  were  encouraged  to  think-aloud  as  they  used 
LATIST  while  completing  scenarios  provided  for  the  test.  Their  thoughts  were  captured  using  a 
tape  recorder;  in  some  cases  supplementary  notes  were  taken. 

The  GMU  Immersion  team  used  a  method  of  problem  severity  classification  to  evaluate 
and  analyze  data  gleaned  from  the  phase  II  usability  tests.  This  approach  treats  problem  severity 
as  a  combination  of  two  factors  -  the  impact  of  the  problem  and  the  frequency  of  users 
experiencing  the  problem  during  the  evaluation.  Survey  results  were  synthesized  for  each 
question.  Common  themes  per  questions  were  tabulated  and  the  results  were  analyzed  to  identify 
potential  changes  to  LATIST.  Based  upon  these  results,  a  list  of  modifications  and  fixes  were 
submitted  to  the  developer  for  the  next  iteration  of  LATIST. 
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Summary  of  Completion  Rate  and  Error-free  Rate 

The  following  tables  summarize  areas  of  success  and  areas  for  enhancement  based  on  the 
completion  rate  and  error  free  rate  measurements: 


Component  Area 


|  Component 

Area 

Explore 

Research 

Participants  located  general  information  about  the  specific  technologies  i.e. 
definitions,  best  practices,  advantages,  disadvantages,  and  recent  reports 

Select  Best 
Technology 

Participants  navigated  through  the  factors  grid  and  located  technologies  that 
worked  given  specific  constraints  i.e.  budget 

Select  Best 
Technology 

Participants  located  recommended  technologies  based  upon  a  specified  learning 
objective 

Apply 

Participants  successfully  located  the  real-world  examples  tab  to  determine  how  a 
given  technology  is  being  used  in  other  government  agencies 

Apply 

Specific  examples  of  the  various  technologies  were  located  by  the  participants 

Global 

Participants  were  able  to  navigate  across  the  multiple  components  of  the  LATIST 
prototype 

Table  3.  Usability  testing  areas  of  success 

Explore 

Research 

Participants  had  difficulty  locating  the  summary  report,  which  displays  all  relevant 
information  regarding  a  specific  technology. 

Select  Best 
Technology 

Participants  had  difficultly  navigating  from  the  factors  grid  to  the  decision  aide. 

Apply 

Participants  had  difficulty  locating  the  area  of  the  tool  that  directs  users  where  they 
can  practice  specific  technologies. 

Table  4.  Usability  testing  areas  for  enhancement 

Component  Area 
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Test  Completion  Time 

The  time  it  took  participants  to  complete  four  scenarios  may  be  an  indicator  of  the  value 
on  investment  for  DAU.  The  average  time  for  completion  of  the  four  scenarios  was  21  minutes. 
In  that  amount  of  time  the  participants  completed  the  following: 

•  Located  Literature  on  blogs 

•  Located  a  Printable  Summary  Report  on  blogs 

•  Located  a  Video  on  How  to  Implement  blogs  into  DAU’s  learning  environment 

•  Located  Real  World  Examples  of  how  blogs  are  being  used  in  other  government  sectors 

•  Located  free  blogging  platforms  to  afford  users  the  opportunity  to  create  their  own  blog 
before  trying  to  use  one  in  their  course 

•  Located  Advantages,  Disadvantages,  and  Best  Practices  of  blogs 

•  Discovered  within  LATIST,  how  to  appropriately  align  learning  objectives  with  learning 
strategies  and  appropriate  technologies 

•  Discovered  within  LATIST,  how  to  identify  which  technologies  are  best  suited  for  use 
based  on  restrictive  factors 

•  Completed  a  scenario  of  an  individual  coming  to  LATIST  to  find  out  what  technologies 
might  be  best  to  use  while  creating  a  new  course  with  the  desire  to  support  a  particular 
learning  strategy. 

The  test  concluded  with  an  open-ended  question,  “What  is  your  overall  opinion  of 
LATIST  in  terms  of  its  value  to  you?”  which  produced  a  variety  of  responses. 

•  Overall  the  tool  received  positive  feedback  and  many  subjects  thought  the  tool  would  be 
helpful  in  raising  awareness  of  technology  options  among  DAU  staff. 

•  In  addition,  the  tool  was  called  a  “good  idea  generator”  and  many  subjects  thought  that, 
with  further  development,  LATIST  would  be  a  good  resource. 

•  Subjects  responded  favorably  to  the  tool’s  use  of  videos  and  many  suggested  that  more  of 
these  multi-media  types  of  resources  be  included. 

•  Subjects  indicated  a  desire  for  more  examples  explaining  how  to  integrate  technology 
into  learning  assets. 

•  Many  subjects  wanted  to  share  the  information  in  the  tool  and  suggested  that  easier  ways 
of  sharing  be  included,  such  as  social  media  links  and  email  options. 

•  The  importance  of  scalability  was  mentioned  several  times  and  issues  of  maintainability 
and  information  relevance  were  stressed  by  several  subjects. 

•  A  desire  for  more  interactivity  between  components  and  between  the  Decision  Aide  and 
the  Factors  grid  were  also  expressed. 
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Recommendations 

LATIST  is  an  electronic  performance  support  tool  designed  to  allow  the  user  to:  (1) 
explore  what  the  research  says  about  a  technology  such  as  advantages  and  disadvantages;  (2) 
select  a  best  technology  for  user  conditions  such  as  learning  objectives  and  bandwidth 
constraints;  and  (3)  review  and  learn  how  to  apply  a  selected  technology.  The  initial  project  goal 
as  discussed  previously  in  this  document  was  to  identify  a  process  that  will  effectively  integrate 
technology  into  DAU  learning  assets  that  enhance  learning.  The  tool  was  conceptualized  as 
research-based  framework  to  assist  DAU  with  the  integration  of  ALT  into  their  learning  assets. 

A  usage-centered  design  approach  was  used  to  meet  the  specific  needs  of  all  users,  regardless  of 
their  role  in  the  ALT  integration  process.  The  results  of  these  efforts  are  reflected  in  the  LATIST 
prototype.  In  order  to  ensure  the  prototype  aligned  with  the  client  needs  and  expectations,  the 
GMU  Immersion  team  conducted  two  rounds  of  usability  testing  on  LATIST 

The  results  of  the  usability  testing  indicated  the  prototype  received  positive  feedback  and 
many  participants  thought  the  tool  would  be  helpful  in  raising  awareness  of  technology  options 
among  DAU  staff.  In  addition,  the  tool  was  called  a  “good  idea  generator”  and  many  participants 
thought  that,  with  further  development,  LATIST  would  be  a  good  resource.  Subjects  responded 
favorably  to  the  tool’s  use  of  videos  and  many  suggested  that  more  of  these  multi-media  types  of 
resources  be  included.  Participants  indicated  a  desire  for  more  examples  explaining  how  to 
integrate  technology  into  learning  assets.  Many  participants  wanted  to  share  the  information  in 
the  tool  and  suggested  easier  ways  of  to  do  so,  such  as  social  media  links  and  email  options.  The 
importance  of  scalability  was  mentioned  several  times  and  issues  of  maintainability  and 
information  relevance  were  stressed  by  several.  A  desire  for  more  interactivity  between 
components  and  between  the  Decision  Aide  and  the  Factors  Grid  were  also  expressed.  While  the 
results  of  the  usability  testing  on  the  prototype  appear  favorable,  the  results  are  based  on  a 
limited  capability  due  to  the  minimal  content  and  functionality  available  in  the  prototype. 

Based  on  comments  recorded  throughout  the  usability  testing  of  LATIST,  the  GMU 
Immersion  team  makes  these  further  recommendations  to  assist  DAU  with  the  continued 
development  of  LATIST: 

1)  Implementation  Recommendations: 

•  Migrate  LATIST  prototype  from  Bean  Creative  servers  to  DAU  servers 

•  Continue  to  develop  more  content  to  give  the  prototype  a  robust  capability 

•  Conduct  subsequent  rounds  of  usability  testing  in  order  to  gauge  the  effectiveness  and 
efficiency  of  the  support  tool 

•  Conduct  further  testing  with  a  greater  diversified  audience  to  verify  functionality  and 
usability  of  the  tool.  Develop  a  content  update  and  maintenance  plan  to  support  relevancy 
and  currency. 

•  Build  out  the  definition,  help,  and  advanced  search  functions 
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•  Develop  login  functionality  so  that  users  may  rate,  share,  and  upload  resources,  take 
notes,  mark  favorites,  and  subscribe  to  an  RSS  feed 

2)  Enhancement  Recommendations: 

•  For  the  Select  Best  Technology: 

o  Enhance  the  Decision  Aide  so  a  user  can  enter  a  specific  technology  and  the 
system  will  output  all  recommended  instructional  strategies  and  corresponding 
learning  levels 

o  Combine  the  Decision  Aide  and  Factors  Grid.  Technology  selected  from  the 
Factors  Grid  should  carry  over  to  the  Decision  Aide,  and  vice-versa. 

o  Add  definitions  to  terminology  in  the  Factors  Grid  as  well  as  in  the  Dictionary. 

•  Explore  Research: 

o  All  cited  references  should  link  to  the  article  (or  to  the  DAU  library  portal  to 
access  the  journal  articles) 

o  Develop  an  Amazon-style  ‘recommended  reading’  feature  based  on  prior  user 
selections 


DAU  stakeholders  indicated  in  usability  testing  that  a  well-organized  and  comprehensive 
performance  support  tool  is  highly  valued  by  DAU.  Implementing  the  aforementioned 
recommendations  and  continuing  to  work  with  users  will  ensure  the  continued  development  of  a 
user-centered  performance  support  tool  to  help  DAU  faculty  and  staff  pedagogically  integrate 
appropriate  technology  into  their  learning  assets. 
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Appendix  A:  Sample  Scenario  and  Use  Case 

Mary  Jane,  a  course  manager,  received  a  mandate  from  her  supervisor  to  incorporate 
wikis  into  one  of  her  courses.  Mary  Jane  is  not  knowledgeable  about  wikis  so  she  decides  to  use 
LATIST  to  learn  more  about  them.  She  starts  with  the  Explore  Research  component  because  that 
area  provides  extensive  research  about  all  technologies.  She  starts  by  viewing  an  overview  of 
wiki  technology.  After  reading  that,  she  decides  she  wants  to  know  about  how  to  incorporate 
wikis  into  the  classroom  environment.  She  finds  an  article  that  says  how  effective  wikis  were  for 
George  Mason  University  (GMU).  The  article  includes  a  link  to  the  GMU  website  that  contains 
more  information.  She  visits  the  GMU  website  and  learns  more  about  this  example 
implementation.  Mary  Jane  really  found  this  article  useful  so  she  marks  it  as  a  favorite  for 
individual  reference  but  she  also  rates  it  highly  so  others  will  know  that  this  article  is  worth 
reading.  Now  she  wants  to  explore  how  other  academic  environments  have  used  wikis.  After 
skimming  a  couple  of  articles,  she  is  convinced  a  wiki  could  be  used  effectively  in  one  of  her 
courses.  Mary  Jane  will  need  to  understand  what  the  best  practices  are  for  this  technology.  She 
also  reviews  the  table  of  advantages  and  disadvantages.  With  an  understanding  of  best  practices, 
she’s  ready  to  see  a  demo.  After  reviewing  a  demo,  she  decides  to  print  the  best  practices, 
advantages,  and  disadvantages  for  easy  reference  while  she’s  reviewing  her  course  plan  to 
identify  areas  where  a  wiki  would  be  most  beneficial. 

These  desires  and  system  responses  were  expanded  to  better  represent  specific  user 
actions  with  the  corresponding  system  responses  as  shown  in  Table  1. 


Course  Manager  Integrates  Mandated  Technology  -  Use  Case 


User  Action 

System  Response 

Enter  Explore  Research 

Display  options:  EDUCAUSE, 

Best  Practices,  Advantages  / 

Disadvantages,  Research,  Personal 

Choose  EDUCAUSE 

Displays  technology  types 

Enters  technology  (wiki) 

Displays  article 

Reads  article 

Closes  article 

Closes  article 

Return  to  choose  another  option 

Displays  options:  EDUCAUSE, 

Best  Practices,  Advantages  / 
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Disadvantages,  Research 

Choose  Research 

Displays  data  fields  to  filter 

articles 

Enters  data  fields  (wiki  case 
history) 

Displays  filtered  articles 

Selects  one  article 

Displays  article 

Reads  article 

Activates  link  to  external  website 

Displays  external  website  in  new 
window 

Reads  external  website  article 

Closes  external  website 

External  website  window  closes 

Closes  article 

Close  article 

Marks  article  as  personal  favorite 

Marks  and  retains  list  of  favorite 

articles 

Rates  article  highly 

Tracks  and  visually  presents 
article  rating 

Selects  another  article 

Displays  article 

Reads  article 

Closes  article 

Closes  article 

Return  to  choose  another  option 

Displays  options:  EDUCAUSE,  ... 

Choose  Best  Practices 

Displays  Best  Practices 

Reads  best  practices 

Prints  best  practices 

Prints  best  practices 

PAGE  36 


LATIST  DESIGN  DOCUMENT 


Closes  best  practices 

Closes  best  practices 

Return  to  choose  another  option 

Displays  options:  EDUCAUSE,  ... 

Choose  Advantages  and 
Disadvantages 

Displays  Adv/Disadv 

Reads  Adv/Disadv 

Prints  Adv/Disadv 

Closes  Adv/Disadv 

Prints  Adv/Disadv 

Closes  Adv/Disadv 

Return  to  choose  another  option 

Displays  options:  EDUCAUSE,  ... 

Choose  Demo  Component 

Displays  transfer  options  (e.g., 
enter  wiki  is  the  technology?) 

Exit  Explore  Research 

Goes  to  a  main  menu  to  select 
Explore  Research,  Select,  or  Demo 

Exit  LATIST 

Closes  LATIST 
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Appendix  B:  Decision  Aide  Data 

Note:  All  associated  data  is  subject  to  change  and  needs  to  be  updatable. 


Level 

Level 

Objectives 

Example 

Objectives 

Instructional 

Strategy 

Instructional 

Strategy 

Definition 

Technology 

Technology  Definition 

Remembering 

and 

Understanding 

Exhibit 
memory  and 
demonstrate 
understanding 
of  facts  and 
ideas,  (define, 
repeat,  explain, 
discuss) 

Example  1: 

From 

ACQ  201A 

Module  2.6: 

Given 

programming 
and  budgeting 
documents,  relate 
the  applicable 
funding  policies 
to  each  of  the  six 
DoD 

appropriation 
categories  of 
greatest  interest 
to  acquisition 
programs 

Example  2: 

From  ACQ  201 A 
Module  2.3: 
Identify  the  four 
steps  of  the  risk 
management 
process  model 

Presentation  / 
Lecture 

In  presentation  / 
lecture,  a  speech  is 
read  or  delivered 
before  an  audience 
or  class,  especially 
for  instruction  or 
to  set  forth  some 
subject. 

podcast, 

vodcast 

A  podcast  is  an  audio  media  file 
available  for  download  from  the 
internet  or  to  subscribers  through 
a  syndication  feed  which  can  be 
played  at  a  later  point  in  time  on 
any  device  capable  of  playing 

MP3  files. 

Questioning 

Provide  students 
with  questions  that 
will  test 

understanding  of 
concepts  and  facts. 

Instructor 
lead  - 

however  the 
instructor 

can  use 
technology 
tools  and 
software  for 
prompts  such 
as  Word, 
PowerPoint, 
Articulate 

Examples 

Ideas  or  objects 
drawn  from  a 
group  of  ideas  or 
objects  to 
represent  core 
features  of  the 
group  from  which 
they  are  drawn. 

Learning 

Management 

Systems 

A  learning  management  system 
(commonly  abbreviated  as  LMS) 
is  a  software  application  for  the 
administration,  documentation, 
tracking,  and  reporting  of 
training  programs,  classroom 
and  online  events,  e-learning 
programs,  and  training  content 

Practice 

In  learning 
cognitive  skills, 
repeated  exposure 
to  concepts 
through  practice 
increases  the 
probability  that  the 
student  will 
remember. 

games, 

simulation 

Simple,  easy  to  play,  short 
games.  Casual  games  are  simple 
to  learn,  inexpensive  to  build 
and  have  a  short  playing  time. 
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Level 

Level 

Objectives 

Example 

Objectives 

Instructional 

Strategy 

Instructional  Strategy 
Definition 

Technology 

Technology  Definition 

Apply  and 
Analyze 

Apply 

knowledge  of 
facts,  concepts, 
and  principles 
to  new 

situations. 

(practice, 

demonstrate, 

compare, 

analyze) 

Example  1: 

From  ACQ201B 
Module  3.4:  The 
student  will  be  able 
to  analyze 
technical  data  to 
identify  risks  and 
ensure  a  system 
will  be 
operationally 
effective  and 
suitable. 

Example  2: 

From  ACQ201A 
Module  4.2:  Apply 
one  or  more 
selected  qualitative 
tools  (e.g., 
fishbone  diagram) 
to  resolve  a 
problems 

Problem 

solving 

Problem-solving 
activities  place  more 
emphasis  on  learning 
how  to  learn,  rather 
than  specific  content.  In 
problem-solving 
activities,  the  process 
of  problem  solving 
becomes  more 
important,  such  as  the 
learner's  ability  to  form 
a  hypothesis,  find  and 
sort  information,  think 
critically  about 
information,  ask 
questions,  and  reach  a 
resolution  or  solution. 

augmented 

reality 

Virtual  data  such  as  text, 
pictures,  and  video  is  combined 
with  what  we  see  in  the  real 
world,  for  the  purpose  of 
enhancing  the  information  we 
can  perceive  with  our  senses. 

Role-play 

activities 

Role-playing  allows 
learners  to  bring  their 
own  experiences  into 
the  role-playing 
situation  and 
consequently  gain 
"ownership"  of  the 
learning  process.  Often 
the  learning 
environment  is 
fictitious  or 
metaphorical  but  also 
engaging  so  it 
captivates  the  learner's 
attention.  Role-playing 
strategies  encourage 
students  to  gain  the 
accompanying 
knowledge  and  skills  in 
order  to  survive  in  their 
"role"  within  the 
learning  environment. 
Skills  could  include 
communication  and 
interpersonal,  key  to 
effective  participation 
in  an  online  learning 
environment. 

simulation, 
virtual  world, 
game 

Virtual  Worlds  allow 
manipulation  of  both  time  and 
space;  slowing  or  speeding 
time;  viewing  microscopic  as 
macroscopic  and  vice  versa. 
Mistakes  can  be  made  without 
penalty  or  consequence  as  in  the 
real  world. 

Articulation 

Articulation  involves 
students  making  tacit 
knowledge  explicit  by 
explaining  to  others 
what  they  know.  As 
students  articulate  their 
knowledge  to  one 
another,  they  share 
multiple  perspectives 
and  generalize  their 
knowledge  so  that  it  is 
applicable  in  different 
contexts. 

blogs, 

microblogging 

Blogs  (or  weblogs)  are  an 
online,  chronological  collection 
of  personal  commentary  and 
links  that  serve  to  capture 
thoughts  and  comments  on  a 
public  website  for  others  to  read 
and  respond  to.  A  blog  is 
frequently  updated  with  entries 
arranged  in  reverse 
chronological  order.  Entries  can 
include  text,  hyperlinks,  images, 
or  multimedia.  Visitors  can  read 
postings,  submit  comments, 
find  entries  by  date  and  by 
keyword. 
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Level 

Level 

Objectives 

Example 

Objectives 

Instructional 

Strategy 

Instructional  Strategy 
Definition 

Technology 

Technology  Definition 

Evaluate 
and  Create 

Propose 
alternative 
solutions  and 
make 
judgments 
about  ideas, 
(plan,  design, 
assess, 
estimate) 

Example  1 

From  ACQ201B 
Module 

2.1:  The  student 
will  be  able  to 
modify,  present, 
and  defend  an 
acquisition 
strategy  to 
accommodate  a 
change  in 
program  funding 
levels 

Example  2 

From  ACQ201B 
Module  1.3:  The 
student  will  be 
able  to  evaluate 
alternative 
approaches  to 
meet  a 
requirement 
based  on 
affordability, 
schedule,  and 
technical 
considerations 

Reflections 

Reflection  involves 
students  reviewing 
what  they  have  done, 
analyzing  their 
performance,  and 
comparing  it  to  that  of 
experts  and  peers. 

Blogs 

Blogs  (or  weblogs)  are  an 
online,  chronological  collection 
of  personal  commentary  and 
links  that  serve  to  capture 
thoughts  and  comments  on  a 
public  website  for  others  to  read 
and  respond  to.  A  blog  is 
frequently  updated  with  entries 
arranged  in  reverse 
chronological  order.  Entries  can 
include  text,  hyperlinks,  images, 
or  multimedia.  Visitors  can  read 
postings,  submit  comments,  find 
entries  by  date  and  by  keyword. 

Hypothesis 

generation 

Hypothesis  generation 
is  a  learning  strategy  in 
which  learners  acquire 
concepts  by  setting 
forth  tentative 
hypotheses  about  the 
attributes  that  seem  to 
define  a  concept  and 
then  testing  specific 
instances  against  these 
hypotheses.  Testing 
and  exploration  offer 
learners  an  opportunity 
to  refine  their 
hypotheses. 

game,  virtual 
world 

A  strategy  game  provides 
players  the  opportunity  to  test 
their  skills  and  theories  in  a 
digital  play  environment  with 
competitive  activities  to  test  out 
hypotheses  and  win. 

Case  study 

Case  studies  are  real 
life  problems  that  have 
arisen  in  the  workplace 
that  students  must 
solve.  Can  also  be 
used  to  explore 
interpersonal 
relationships. 

simulations, 

videos 

A  simulation  is  a  computer 
generated  model  that  imitates  a 
real-life  scenario  with  variables 
and  options. 

Collaboration 

In  collaboration  and 
social  negotiation,  the 
goal  is  to  share 
different  viewpoints 
and  ideas  and  to 
collaborate  on 
problem-solving  and 
knowledge  building 
activities.  Groups  are 
formed  to  provide 
variation  in  classroom 
activity  (face-to-face 
or  virtual),  share 
workloads  (permitting 
larger  projects),  and 
promote  peer  tutoring. 

wikis,  social 
network 

Wikis  are  editable  websites 
where  any  user  can  edit,  add,  or 
read  content  to  that  particular 
website.  Their  structure  and 
content,  be  it  text  pictures,  or 
other  content  format  can  be 
changed  readily.  Wikis  support 
collaboration,  problem  solving, 
critical  thinking  and  may  also 
motivate  students. 

PAGE  40 


LATIST  DESIGN  DOCUMENT 


Appendix  C:  LATIST  Design  Requirements 

The  design  requirements  for  LATIST  are  described  from  both  a  system  and  user 
perspective. 

LATIST  System  Requirements 

The  LATIST  tool  will  be  implemented  utilizing  a  WordPress  platform  with  MySQL  as 
the  database.  Also  the  LATIST  system  must  effectively  work  with  Internet  Explorer  versions  7.0 
and  above  in  a  Windows  2003  .Net  Framework.  Additionally,  the  system  should  be  built 
cognizant  of  Section  508  requirements,  however  full  508  compliance  can  be  incorporated  after 
the  prototype  has  been  approved  by  DAU.  DAU  prefers  SQLServer  2005  but  will  accept 
MySQL. 

When  designing  the  LATIST  system  there  are  several  considerations  which  must  be 
accounted  for  in  the  working  prototype.  These  include: 

•  DAU  logo  (Link  for  instructions:  http://www.dau.mil/pubscats/Pages/MediaKit.aspx) 

•  GMU  logo  (Link  for  instructions:  http://logo.gmu.edu/webguide/) 

•  Users  must  be  able  to  access  the  desired  information  or  component  within  three  clicks. 

•  Download  and  navigation  response  time  should  be  as  fast  as  possible,  with  a  goal  not  to 
exceed  10  seconds  for  downloads.  (Note:  This  assumes  high  speed  internet ) 

•  The  development  platform  consisting  of  hardware  and  software  requirements  will 
conform  to  DAU  standards. 

•  Design  factors  must  include  use  of  the  tool  on  mobile  devices. 

•  LATIST  will  support  multiple  navigation  points.  Users  can  navigate  and  search  by 
component  or  by  technology. 

•  LATIST  will  be  linked  to  DAU.mil  and  should  follow  the  www.dau.mil  color  schemes. 

•  Screen  Resolution  for  PC's  and  mobile  devices  must  meet  standards  for  maximum 
usability. 

LATIST  User  Requirements 

This  section  summarizes  the  user  requirements,  flowchart,  and  wireframe  for  LATIST 
from  a  global  perspective.  The  following  table  lists  and  describes  the  features  and  functionalities 
available  to  the  users. 
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Print 

The  user  shall  be  able  to: 

•  print  documents  in  their  native  format 

•  print  screen  in  a  print  friendly  version 

•  print  text  versions  of  all  audio 

Save 

The  user  shall  be  able  to  save: 

•  information  in  its  native  format  to  a  user  specified  location 

•  a  report 

•  a  webpage 

•  a  media  fde 

Navi 

gate 

The  user  shall  be  able  to  seamlessly  navigate  to  any  component  within 
LATIST. 

Search 

The  user  shall  be  able  to: 

•  search  across  LATIST 

•  search  by  keyword 

Dictionary 

The  user  shall  be  able  to  : 

•  Interactively  search  for  term  definitions  related  to  LATIST  and 
technologies 

Login 

The  user  shall  login  in  to  modify  LATIST  content: 

•  to  upload,  take  notes,  mark  as  favorite,  rate,  view  favorites,  save 

Help 

The  user  shall  be  able  to  request  help  for  technical  issues. 
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Appendix  D:  Explore  Research  Component  User  Requirements  and  Flowchart 
Explore  Research  Flowchart 

The  primary  navigation  for  LATIST  will  be  by  component  (Explore  Research,  Select 
Best  Technology,  Apply  Technology)  while  a  secondary  navigation  by  technology  will  be 
supported  (Technology  quick  links).  Having  selected  Explore  Research  from  the  primary 
navigation,  the  user  selects  a  technology  to  research.  Once  a  technology  is  selected,  the  user  is 
presented  with  an  overview  of  that  technology.  The  user  will  be  able  to  select  to  review 
advantages,  disadvantages,  best  practices,  and  literature  on  that  technology.  After  the  user  has 
selected  a  technology  and  reviewed  the  provided  research-based  information,  the  user  can  move 
to  another  component  based  on  that  technology.  For  example  the  user  can  seamlessly  navigate  to 
the  Apply  Technology  component  (tutorial  or  practice)  with  the  technology  already  selected. 
These  relationships  are  indicated  in  the  flowchart. 


Explore  Research  Flowchart 
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□EE3 

Register  sr  Login 


Search 

Advanced  Se-ajch 

Dictionary 


Explore  content 


.About  this  croiectl  Helo 

DAU  1  GMU  1  Bean 

Explore  Research  Home  Page 


from  Apply 

Decision  WkSget 

Fa  stars-  Wi  dget 

LATIST 


Explore 

Select  1 1  Apply 

Technologies 

Explore  Research  User  Requirements 

The  following  outlines  the  requirements  for  the  prototype.  Each  requirement  is  noted  as 
‘must  have,’  mock-up,  or  later  development;  ‘must  have’  items  are  intended  to  be  programmed; 
mock-up  items  need  only  have  the  appearance  of  functionality;  and  later  development  items  do 
not  need  to  be  addressed  unless  it  impacts  the  required  and  mock-up  functionality. 

While  in  the  Explore  Research  component,  the  user  shall  be  able  to  navigate  to  any  other 
Explore  Research  subcomponent  or  the  Select  Best  Technology  component  or  the  Apply 
Technology  component  or  exit  LATIST.  The  user  should  be  able  to  add,  upload,  search,  take 
notes,  rate  resources,  mark  as  favorite,  print,  and  subscribe/receive  RSS  feeds,  and  print.  Note 
that  any  modification  to  LATIST  content  will  require  user  login.  These  functionalities  are  defined 
below. 


•  Add:  The  user  shall  be  able  to  add  content  to  the  advantages,  disadvantages,  and  best 
practices  tables,  (mock-up) 

•  Search:  The  user  shall  be  able  to  search  for  resources  across  LATIST  by  author,  date, 
title,  technology,  sub-technology,  keyword,  rating,  and  marked  as  personal  favorites, 
(must  have) 
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•  Search  Results  -  The  search  results  include  the  author,  date,  title,  average  rating,  access 
the  full  citation,  link  to  the  resource,  link  to  the  user’s  personal  notes  associated  with  the 
resource  and  if  the  user  has  previously  marked  the  resource  as  a  favorite.  The  user  shall 
be  able  to  sort  search  results  by  author,  date,  and  rating.  Note  that  not  all  fields  may  be 
populated  for  all  resources.  The  user  shall  be  able  to  view  /  read  selected  resource  in  its 
native  format  (pdf,  excel,  wmv,  etc). 

•  Upload:  The  user  shall  be  able  to  upload  resources  and  provide  database  elements 
(author,  date,  title,  author,  rate,  file  name;  see  data  element  model).  Add  a  check  box 
reflecting  user  has  reviewed  copyright  permission. 

•  Notes:  The  user  shall  be  able  to  create  personal  notes  for  each  resource  and  as 
independent  files.  The  notes  can  be  edited,  saved,  and  printed. 

•  Rate:  The  user  shall  be  able  to  enter  a  rating  for  any  resource;  an  average  for  all  users  will 
be  displayed  visually  with  an  iconic  rating  of  1  -  4. 

•  Mark  as  Favorite:  The  user  shall  be  able  to  mark  as  favorite  any  resource  from  the  search 
results  list. 

•  Review  Favorites:  The  user  shall  be  able  to  view  their  list  of  personal  favorites.  The  list 
can  be  presented  similar  to  the  search  results  and  with  the  same  functionality. 

•  Print:  The  user  shall  be  able  to  print  resources  in  its  native  format  and  print  screen  in 
printer  friendly  format,  such  as  the  results  from  the  search  function. 

•  RSS  Feeds:  The  user  shall  be  able  to  subscribe  to  and  receive  RSS  feeds.  An  RSS  will  be 
developed  that  notifies  users  to  resources  uploaded  to  LATIST. 

•  Save:  The  user  shall  be  able  to  save  selected  resources  to  a  user  specified  location. 

•  Recommended  Reading:  The  user  shall  be  able  to  view  a  recommended  reading  list  based 
on  prior  resource  selection. 
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Appendix  E:  Select  Best  Technology  Component  User  Requirements  and  Flowchart 

Generate  report:  Through  both  the  Decision  Aide  and  the  Factors  Grid,  users  shall  be  able 
to  generate  a  report  about  a  specific  technology.  This  report  will  include: 

•  Short  description  of  technology 

•  List  of  strategies  supported  by  technology 

•  Factors  mapped  to  technology 

•  Links  to  technology  sections  of  Explore  Research  and  Apply  Technology  components 

•  Links  to  print,  save,  and  share 
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Print  report:  The  user  shall  be  able  to  print  the  report  in  a  print-friendly  format. 

Save  report:  The  user  shall  be  able  to  save  the  report  to  a  user  specified  location. 

Share  report:  The  user  shall  be  able  to  share  the  report  through  email. 

Browse  factors:  The  user  shall  be  able  to  browse  information  in  the  factors  matrix  by 
technology  or  by  factor. 
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Navigate:  The  user  shall  be  able  to  navigate  to  the  other  components  of  LATIST  that  are 
related  to  a  specific  technology. 
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Select  Best  Technology  Flowchart 
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Appendix  F:  Apply  Technology  Component  User  Requirements  and  Flowchart 
Apply  Technology  Flowchart 

The  primary  navigation  for  LATIST  will  be  by  component  (Explore  Research,  Select 
Best  Technology,  Apply  Technology)  while  a  secondary  navigation  by  technology  will  be 
supported  (Technology  quick  links).  Having  selected  Apply  from  the  primary  navigation,  the 
following  describes  the  layout  of  the  Apply  Technology  component.  Navigation  within  the 
Apply  Technology  component  will  be  by  the  technology  classes:  1)  Social  Media,  2)  Mobile 
Technology,  and  3)  Virtual  Worlds/Games  and  Simulation.  Each  technology  class  will  have 
specific  technologies  listed  within  that  category.  Each  technology  will  have  the  following 
functions: 

•  How  to  implement  subcomponent:  In  this  area,  the  user  could  read,  view  or  link  to  an 
explanation  of  how  to  implement  a  specific  technology  contextualized  to  DAU  content. 
The  subcomponent  will  contain  Flash/video/text/text  document/URL  files  as  required  to 
access  the  how  to  develop  explanation. 

•  Real  world  examples  subcomponent:  In  this  area  the  user  will  be  provided  with  a  URL  to 
an  external  website  which  will  allow  them  to  get  an  opportunity  to  see  how  a  specific 
technology  is  being  used  in  a  real  world  context  as  well  as  contextualized  to  DAU 
content. 

•  Practice  subcomponent:  This  area  will  provide  the  user  a  URL  to  an  external  website 
which  will  allow  them  to  get  an  opportunity  to  experiment  using  a  specific  technology. 

These  relationships  are  indicated  in  the  flowchart  below.  The  basic  wireframe  developed 
for  the  Apply  Technology  component  based  on  the  flowchart  is  also  provided. 
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Apply  Flowchart 
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Apply  User  Requirements 

The  following  outlines  the  requirements  for  the  prototype.  Each  requirement  is  noted  as 
‘must  have,’  mock-up,  or  later  development;  required  items  are  intended  to  be  programmed; 
mock-up  items  need  only  have  the  appearance  of  functionality;  and  later  development  items  do 
not  need  to  be  addressed  unless  it  impacts  the  ‘must  have’  and  mock-up  functionality. 

While  in  the  Apply  Technology  component,  the  user  shall  be  able  to  navigate  to  any  other 
Apply  subcomponent  or  the  Select  Best  Technology  component  or  the  Explore  Research 
component  or  exit  LATIST.  The  user  should  be  able  to  add,  upload,  search,  print,  send,  and 
login.  Note  that  any  modification  to  LATIST  content  will  require  user  login.  These  functionalities 
are  defined  below.  Note:  not  all  functionalities  are  represented  in  the  prototype. 

•  Add:  The  users  shall  be  able  to  add  content  to  any  of  the  subcomponents  within  the 
Apply  Technology  component. 

•  Search:  The  user  shall  be  able  to  search  for  resources  across  LATIST  by  author,  date, 
title,  technology,  sub-technology,  keyword,  rating,  and  marked  as  personal  favorites. 
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•  Upload:  The  users  shall  be  able  to  upload  resources  and  provide  database  elements 
(author,  date,  title,  author,  rate,  file  name;  see  data  element  model). 

•  Print:  The  users  shall  be  able  to  print  resources  in  its  native  format  and  print  screen  in 
printer  friendly  format,  such  as  the  results  from  the  search  function. 

•  Save:  The  users  shall  be  able  to  save  selected  resources  to  a  user  specified  location,  in 
any  specified  format  and  file  type. 

•  Navigate:  The  users  shall  be  able  to  navigate  anywhere  within  LATIST  within  3  clicks  or 
less;  users  shall  be  able  to  seamlessly  navigate  to  subcomponents  within  the  Apply 
Technology  component  as  well  as  to  related  content  within  other 

components/ subcomponents . 

•  Send:  The  users  shall  be  able  to  send  from  the  Apply  Technology  component.  Users  will 
need  the  ability  to  send  media  files,  emails,  and  links  to  pages  and  print  friendly  versions 
of  text  from  the  Apply  Technology  component. 

•  Open  Content:  The  users  shall  be  able  to  easily  open  all  content  in  the  Apply  Technology 
component.  Users  shall  be  able  to  open  all  content  using  double  right  clicks  in  standard 
windows  format  with  the  option  of  opening  material  in  a  separate  window. 

•  Login:  The  users  shall  be  able  to  login  prior  to  adding  or  uploading  content  into  the 
Apply.  Users  will  be  able  to  login  using  an  ID  and  password,  through  creating  an  ID  and 
password  for  the  first  time,  and  shall  be  able  to  access  all  secure  elements  of  LATIST 
once  logged  in. 
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